Video conferencing with mobile devices is becoming more and more commonplace. However, video captured with a mobile device is often noisy due to the space/size constraints of the video capturing devices on the mobile device.
The video capturing device, e.g., camera, charge-coupled device (CCD), CMOS image sensor, and the like, provided in mobile devices have much smaller image sensors than stand-alone cameras. As a result, when video is captured/recorded on a mobile device, especially in low-light conditions, the resulting images/videos are often noisy.
Although there are various known processes for reducing noise from captured video footage, many of these known video noise reduction filters (VNRs) are not only processor intensive but are not capable of being implemented in real-time interactive applications, such as video conferencing, where low-delay, for example, 200 ms or less, is an essential requirement. Furthermore, many conventional real-time VNR algorithms are codec specific.
Many of the conventional VNRs that are suitable for real-time interactive applications are often embedded in the codec and/or make use of encoding data, for example, motion vectors, computed as part of the coding process. As a result these conventional VNRs are considered internal VNRs. An alternative to conventional internal VNRs is to parse the bitstream of the codec and use the parsed information in an external denoising component. However, this alternative is not universally applicable and depends on the codec (via the parsing), also the parsed information from the bitstream can only be extracted from the previous frame and hence introduces some delay and/or visual artifacts.
Video conferencing platforms/applications may be used in may products, such as Google Hangouts™, Facebook Messenger™, Amazon Mayday™ and Snapchat™. These platforms/application may support several video codecs, for example, VP8, VP9, and H.264. Although some of the standardized codecs, such as, VP8 and VP9, may have internal video noise filters (VNRs), which are embedded in the encoder and use information collected from the encoder, such as motion vectors, hardware codecs are emerging, which normally do not have video noise filters.
Accordingly, a need exists for a codec independent video noise reduction filter (VNR) capable of meeting the processing requirements of video conferencing applications on mobile devices.